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STATE  ESTIMATION  AND  CONTROL  OF  CONDITIONALLY  LINEAR  SYSTEMS* 


WOJCIECH  J.  KOLODZIEJ^  AND  RONALD  R.  M)HLER^’^ 

Abstract .  The  fllterincr  problem  for  a  partially  observable  stochastic  system, 
with  linear  in  observable  states  dynamics  and  non-Gaussian  initial  conditions 
is  studied  here.  It  is  shown  that  the  conditional  expected  value  of  the 
unobservable  states,  given  the  past  observations,  can  be  expressed  in  terms  of 
a  finite  dimensional  set  of  statistics.  This  result,  which  generalizes  the 
conditionally  Gaussian  filter  is  used  to  derive  a  separation  principle  for  a 
linear^quadratlc  control  problem.  ^ 
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Introduction 


Stochastic,  partially  observable  systems,  with  linear-in-observable  state 
dynamics  are  termed  conditionally  linear  systems  here.  It  is  well  known  that 
the  solution  of  a  state  estimation  problem  for  a  conditionally  linear  system 
with  Gaussian  distribution  of  the  initial  state  is  given  in  terms  of  two  sets 
of  sufficient  statistics,  satisfying  stochastic  differential  equations  [4] . 

Solved  here  is  the  state  estimation  problem  which  generalizes  the  above 
result  for  the  case  of  an  arbitrary  a  priori  distribution.  The  method  applied 
in  this  study  is  based  on  the  derivation  of  an  explicit  formula  for  the  condi¬ 
tional  characteristic  function  of  the  state,  given  the  past  and  present 
observations.  This  approach  seems  to  Impose  less  restrictive  conditions  on 
the  system  structure  than  the  methods  based  on  the  derivation  of  the 
conditional  distribution  function.  The  latter  can  be  found  in  [1]  where  the 
filter  is  derived  for  a  linear  system  with  a  priori  distribution  having  a 
well-defined  density  function. 

It  is  shown  here  that  the  conditional  characteristic  function  of  the 
present  and  past  states,  given  the  present  and  past  observations,  is  para¬ 
metrically  determined  by  a  finite  number  of  sufficient  statistics.  This 
result  leads  to  the  derivation  of  a  filter,  in  the  form  of  a  finite  set  of 
stochastic  differential  equations  which  extends  the  result  of  [1]  in  a  similar 
manner  as  a  conditionally  Gaussian  filter  generalizes  a  Kalman  filter. 

Also  discussed  here  and  illustrated  by  the  examples,  is  the  suitability 
of  the  filter  structure  for  the  study  of  stochastic  control  and  parameter 
estimation. 

1 .  Problem  Formulation  and  the  Main  Result 

Given  the  following  system  of  stochastic  differential  equations 
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(1.1)  dx^  “  (fQ(t,y)  +  f^(t,y)x^)  dt  +  gQ(t,y)dw^  +  qQ(t,y)dv^, 

(1.2)  dy^  »  (hQ(t,y)  +  h^(t,y)x^)dt  +  dv^ ,  0  <  t  <  T, 

where  fg.f pgQ,qQ,hQ,hj  are  the  nonantlcipative  functionals  of  y  (l.e., 
measurable  with  Tj.  *  o  -  alg  {y^,  0  <  s  <  t}),  and  w^,  are  independent 
Wiener  processes. 

A 

The  objective  Is  to  find  x^  =  E(x^/Y^)  ,  assuming  that  x^,  y^  satisfy 
(1.1)  and  (1.2),  and  that  the  conditional  distribution  of  the  initial  states 
F(a)  -  P(xq  <  ajyg)  is  given. 

The  organization  of  this  section  starts  with  Lemma  1,  whereby  it  is  shown 
that  the  conditional  characteristic  function  of  (Xj-Q*  ’^tl’  ***»  ’^tnH^t* 
an  arbitrary  decomposition  0  <  tQ  <  tj^  <  . . .  <  t^  <  t  <  T,  of  the  interval 
(0,T]  Is  of  a  particular  form.  Results  from  the  theory  of  conditionally 
Gaussian  processes  are  used  here. 

Next,  Lemma  2,  the  explicit  formula  for  the  characteristic  function  of 
XjjTj.  is  derived,  and  finally,  in  Lemma  3,  all  the  results  are  organized  to 
yield  the  recursive,  finite-dimensional  set  of  filter  equations. 

The  assumptions  used  in  the  proof  of  Lemma  1  and  2  are  listed  below: 

Let  Cj,  denote  the  space  of  continuous  functions  n*{iri^.,0<t<T}.  It 
is  assumed  that  for  each  p  e 

T  1 

(1.3)  /  (  I  (|f.  (t,n)|+|h  (t,n)|)+|go(t,n)|^+|qo(t,Ti)|^)  dt  <  » 

0  k-0 

The  above  assimptlon  assures  the  existence  of  the  (Ito)  integrals  in 
(1.1)  and  (1.2)  [3].  In  order  to  use  the  results  for  conditionally  Gaussian 
processes  it  is  also  assumed  that  [4] ; 


(1.4)  for  all  n  e  ,  t  e  (0,T] ,  |f^(t,n)|+|  hj(t,n)|  <  const 

and 

T 

(1*5)  /  E(|f-(t,y)|^+|g-(t,y)l^+jq.(t,y)j^]dt  <  «,  E(|x^|^]  <  • 

0 

Lemma  1 . 

Let 

n 

I 

k-0 

Then  the  conditional  characteristic  function  of  (Xj-Q*  *tl»  •••»  ’^tn^l^t  ^ 
given  by 

m 

(1.6)  e^(z)  -  -  /  exp  [Q(t,a,z,y) )  dF(a) 


♦j.  *  exp(i 


where  Q(t,a,z,y)  is  quadratic  in  the  variables  a  and  z. 
Proof  of  Lemma  1 

First  notice  that  (1.1)  solves  as 


(1.7) 

where 


t 

*t  "  ^^1  " 


Rewrite  (1.7)  in  the  symbolic  way  as 


Now,  the  following  version  of  the  Bayes  formula  will  be  used  [2,  p.  B] : 
Let  (^^(xQ,w,y)  be  a  nonanticipative  functional  of  its  arguments  with 
kUcI  )  <  *  for  all  t  e  I0,T]  .  Then 


(1.9)  E((^^|t^)  “  /  /  (^^(a,n,y)p^(a,n,y)d(j^(n)dF(a) 

“•  c_ 

T 

where  is  a  Wiener  measure  in  the  measurable  space  of  continuous  functions  n 
on  tO,T] , 

t  .  1  2  ■‘2 

Pj.(a,n,y)  -  exp(  /  h^  (t|»g(a,n,y)  -  x^(y)  )dv^  “  Y  /  'll  ('l'g(a ,  n,y)  -  Xg(y))  ds) 

(1.10) 


with  dVg  -  dyg  -  (hg  +  h^  x^)ds,  and  tg(a»h,y)  defined  by  (1.8).  The  random 
process  Vj.  can  be  represented  by 


t 

''t  ”  J  ^‘^^s  “  *  hj(s,y)Xg(y)]ds} 


V  +  /  h  (s,y) (x  -X  (y)  ]ds  . 

c  0  A  s  s 


Now  using  the  Ito  formula  we  have 

Izv  Izv  t  Izv 

e  *  e  ®  +  iz  J  hj(T,y)e  ^  (x^ 

s 


x^(y)  )dT 


t  izv 


2  t  izv 


Multiplying  both  sides  of  the  above  equation  by  e  and  taking  the  condi¬ 
tional  expectation  B(«|t  )  gives 


iz[v  -V  1  2  t  izfv  -V  1 

K(e  -  1  -  f-  /  E(e  '  ^  ^  Vs )  . 

s 


Solving  the  last  equation  yields 


-  |-<t-s) 

(1.11)  K(e  |t  1  -  e 


which  shows  that  (''t'^t^  ®  Wiener  process. 

Now  rewrite  p^(a,Ti,y)  in  a  more  convenient  form.  To  this  end  Introduce  the 
following  notation: 


A^(t,y)  -  h^(*^U  /  C  ^^^s)  " 


A2(t,y)  -  h^*^  , 


AjCt.y)  -  ♦;  gg 


t  1  2 

Cj(t,y)  -  /  Aj(3,y)dVg  -  j  f  Aj(s,y)ds  , 


t  t 

C,(t,y)  -  /  A-(s,y)dv  -  /  A, (s,y) A-(s,y)ds  , 

^  0  ^  “  0  ^  ^ 


C,(t,y)  -  (  /  A^(8,y)ds)^^^  , 


t  s  t  s 

CA(t,y,w)  -  /  A,(s,y)  /  A-(T,y)dw  dv  -  /  A, (s,y) A,(s,y)  /  A,(T,y)dw  ds  , 

0  0^  0  ^ 
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t  3 

r  ^  \  r 


C-(t,y,w)  »  -  /  A_(s,y)  /  A-(s,v)dw  ds 

^  0  0  ^ 


Note  also  that  C^(t,y,w)  and  O^Ct.y.w)  can  be  rewritten  with  the  use  of  the 
Ito  formula  by: 


C.(t,y,w)  =  /  A,(t,s,y)dw  , 


C,-(t,y,w)  =  /  A-(t,s,y)dw 

0  ^  ^ 


where 


t  s 

A.(t,s,y)  -  (  /  A,(s,y)dv  -  /  A,(T,y)dv  ) 

^  0  0  ^ 

t  s 

-  (  /  A, (8,y)A„(s,y)ds  -  /  A  (T,y)A,(T,y)dT))A-(s,y) 

0  0  ^ 


®  2  ^2 

AjCt.s.y)  -  (  /  A2(T,y)dT  -  /  A2(s,y)ds )A^(s,y) 


Now,  usln;;  the  above  notation  we  have  from  (1.8)  and  (1.10) 


Z  t  S 

Pc(a,w,y)  -  exp(C^+  a(C2+  C5)  +  C^-  f"  7  Ajdw^j^ds) 

-  exp(Cj+  aC2-  ^  J  1  J  4^/ 


(1.12) 

The  arguments  in  (1.12)  were  omitted  for  brevity. 
From  (1.8)  it  follows  that 
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Ag(t,y)  +  ^/  A3(s,y)dWg] 


where 


Ag(t,y)  -  J  (fg-  qQhJds  +  J  q^dy^ 


Combining  (1.12)  and  the  above 


exp(Q(t,a,z,y))  =  /  ((i^(a,  n,y)p^(a,  n,y)d  u^(  n) 


■  ""(S”  *V  -r  “(J, 

K»1 


n  t 

k.i  tK  b  fc  n  Q  5  4  s 


^  ^^k*tk  “  i  ^  ^  A.dn  )^ds)du  (n). 

k-10  ^  ^  ^  0  ^  0  ^  ^  ^ 


(1.13) 

In  order  to  evaluate  the  Integral  In  (1.13)  the  following  results  will  be 
used: 


(1)  Since  the  above  Integral  represents  a  conditional  expected  value  of 
Its  Integrand,  under  the  condition  that  yg,  s  e  [0,t]  and  Xq  =  a  are 
given,  the  resulting  distributions  are  of  conditionally  Gaussian 
type  [4].  Note  that  this  fact  does  not  depend  on  the  F(a). 

(11)  With  all  the  variables  In  (1,13)  being  conditionally  Gaussian  we  can 
use  a  convenient  theorem: 


Theorem  [4,  pp.  12-13] 

Let  w^,  t  e  [0,T]  be  a  Wiener  process  and  let  R(t),  G(t) ,  and  H(t)  >  0  he 
such  that 

T 

/  (|R(t)|  +  G(t)^  +  H(t))dt  <  «  . 

0 

Then  for  all  t  e  [0,T] 

t  t  s  » 

(1.14)  E(exp(  /  R(s)G(s)dw  -  /  H(s) (  /  G( T)dw  ]  ds ) 

o  ®  0  0  ^ 
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=  exp(-^  D(t)  +y  /  G(s)  r(s)ds) 

0 

where  dr(s)  -  (2H(s)  -  r(3)^G(s)^)ds  ,  r(t)  =  0  , 

t 

and  D(t)  is  the  covariance  of  /  R(s)dC  ,  where 

0  ® 

dc  -  G(s)^r(s)C  ds  +  G(s)dw  ,  0  . 

s  s  s  0 

Comparing  the  last  Integral  in  (1.13)  with  the  equation  given  by  (1.14), 
we  note  that  the  corresponding  R(t)  is  a  linear  function  of  a  and  z.  Now 
(1.9),  (1.13),  (1.14),  and  the  definition  of  D(t)  conclude  the  proof  of 
Lemma  1 . 

From  Lemma  1  it  follows  in  particular  that  for  z  e  R,  the  characteristic 
function  of  Is  given  by 


in 


(1.15)  e^(z)  =  C(t,y)  /  exp[a  F^(t,y)  +  aF2(t,y)  +  izaF^(t,v)  +  izF^(t,v) 

+  z“F^(t,y) ]dF(a)  , 

where  Fj ,  F2,  F^,  F^,  F5  do  not  depend  on  F(a) .  Normalizing  e^(z)  (i.e., 
requiring  that  6^.(0)  =  1)  yields 


(1.16)  e^(z)  =  expfizF^  +  z^F^)  — 


/  exp[a^Fj^+a(F2+lzF2  }dF(a) 


/  exp(a  F^+aF2}dF(a) 


Then  from  the  general  properties  of  the  characteristic  function,  it  fo  ws 


de^(z)j 


1  dz  z  »  0  t 


1  ?  ^  ®,.(z)  M 

(t)  — V-  .-0 

dz 


where  *  E((x^  -  x^)  |Y^)  i.e.,  the  conditional  variance  of 
From  the  above  and  (1.16) 


(1.17)  Xj.  -  F3l^.(l)  +  F^  , 


(1.18)  Pj.  -  -2F5  +  F3  (lj.(2)  -  1^(1)] 


where 


(1.19)  I^(n) 


J  a”exp(a^Fj+aF2}dF(a) 


/  exp(a  F^+  aF2)dF(a) 


,  n  *  1,2 


The  following  Lemma  defines  F^^  1  *  1, 2, 3, 4, 5  in  (1.16) 


Lemma  2 


The  characteristic  function  of  is  given  by 


(1.20)  e^(z)  *  exp(-  j  z^P^(O)] 


2 

/  exp[a  F^+aF2+izx^(a,0)]dF(a) 

OO 

/  exp[a^Fj+aF2)dF(a) 


where  x^(a,0),  P^(0)  are  given  as  the  solutions  to  the  following  set  of  dif 
ferentlal  equations  with  a  =  0: 


(1.21)  dXj.(a,a)  =  (fQ+f^x^(a.a))dt  +  (q^H-  P,(  a)h  J  [dy^- (hg+  h^x^(a,o)  ]dt  ] 

XQ(a,a)  =  a  , 

(1.22)  dP^(o)  =  (2f^P^(a)  +  g^+  qQ-(qQ+  P^(a)hj^}2)dt  ,  ?^(a)  =  , 


and 


(1.23) 

(1.24) 

(1.25) 


t 

^2-^/  . 

t 

♦c  “  (^r  hjqg+  P^(0)h^j)dsj  . 


Proof  of  Lemma  2 


Since  the  do  not  depend  on  F(a)  (see  Lemma  1)  take 
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In  this  case  the  resulting  conditionally  Gaussian  distribution  allows  for 
explicit  e^(z)  calculation  [4],  Accordingly, 

(1.27)  e^(z)  “  exp(lzx^(m,o)  -  yz^P^Co)]  , 

where  Xj.(m,a)  and  P^(a)  satisfy  (1.21)  and  (1.22)  respectively.  With  F(a) 
given  by  (1.26)  it  follows  from  (1.16)  that 

(1.28)  =  exp(lz(F^+  o^(F2+  ^>3 )  +  7  . 

a 

*-2  -2 

where  a  *  a  -  2Fp 

Comparing  (1.27)  and  (1.28),  we  have 

(1.29)  x^(m,a)  -  F^+  +  ^)f’3  ♦ 

a 

and 

(1.30)  Pj.(o)  =  a^F3  -  2Fg  . 

Letting  now  o  0  in  (1.29)  and  (1.30),  it  follows  that 

(1.31)  F,  +  mF_  =  x^(m,0)  , 


The  above  allows  ej.(z)  to  be  of  the  form  of  (1.20)  with  and  F2  yet  to 
defined.  Using  now  (1.17)  and  (I.IB)  and  explicitly  calculating  I^(n) , 
n  -  1.2, 


(1.33)  -  2fJ  * 


(1.34)  F2}  =  F3(x^  -  Fj  , 


At  =•  -  P^(0)  . 


The  formulae  for  Fj^  and  F2  will  be  obtained  by  differentiating  (1.33)  and 

(1.34).  However,  before  this  is  done  recall  from  the  theory  of  nonlinear 
filtering  [3]  that  in  general  for  x^,  y^.  given  as  a  solution  to  (1,1)  and 

A 

(1.2)  x^,  satisfy 


(1.35)  dx^  =  (fg  +  f^x^jdt  +  (qg  +  Pj.^^  *  ^0  ~  ^  adF(a)  , 


dp^  -  (2fiP,  *  sj  +  qo  -("0  *  •  ’’o 
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Remark.  Direct  application  of  Eqs.  (1.35)  and  (1,36)  meets  the  difficulty  of 
infinite  coupling  between  the  subsequent  moments. 

From  Eqs.  (1.22),  (1.35),  and  (1.36),  and  the  fact  that  for  conditionally 
Gaussian  processes  >  0, 

(1.37)  dA^  .  Aj2f^  -  h^[2qQ  +  hj(P^  +  Pt^0))])dt  , 

Now  from  (1.33)  and  (1.34)  (upon  differentiation)  and  using  (1.35),  (1.37)  it 
follows  that 

(1.38)  =  ■  Y  ^3  dt  ,  F^(0)  =  0  , 
and 

(1.39)  dF2  =  +  h^F2lj.(l)dt)  ,  F2(0)  -  0  . 

To  define  Fj,  notice  that  Eq.  (1,21)  solves  as 

^  -I  -  1  - 

-  ^^(a  +  J  (fg  -  hg(qg  +  Pg(a)hj))ds  +  J  [qg+  Pg(a)hj)dy^)  , 

(1.40) 

where 

t 

(1.41)  (|)j.  -  ®*p(^/  (^r^i(9o'^  Pg(o)hlj)ds 

Comparing  the  above  with  (1.31)  shows  that  F3  >  for  a  -  0,  which  ends  the 


proof  of  Lemma  2 


Lemma  3  below  merely  organizes  all  the  results  into  the  filter  equations 
and  the  final  form  of  the  conditional  characteristic  function. 

Lemma  3 

Given  the  system  (1.1)  and  (1.2)  together  with  the  a  priori  distribution 
F(a)  ■  P(xq  <  a|yg).  The  following  are  the  filter  equations  (i.e.,  formulae 
of  the  recursive  type,  which  calculate  =«  E(x^|t^)). 

00 

(1.42)  dx^  -  (fQ+  +  (qQ+  =■  /  adF(a)  , 

—  00 

(1.43)  dVj.  -  dy^.  -  (hg  +  hj^x^jdt  , 

(1.44)  +  ^J(I^(2)  -  lj(l))  , 

(1.45)  dPj.  »  (2fjPj.  +  g2  +  ^  P^hJ^jdt  .  Pg  -  0  . 

»  *> 

/  a"exp(a  aF2)dF(a) 

(1.46)  I  (n)  -  -  ,  n  -  1,2  , 

t  *2 

/  exp(a  Fj+  aF2)dF(a) 

—  00 

(1.47)  dFj  -  -  Y  hj<.Jdt  ,  F^(0)  -  0  , 

(1.48)  dF2  -  <l»,.hj(dVj.  +  ^^hjl^(l)dt)  ,  F2(0)  -  0  , 

(1.49)  d^^  -  [f^  -  h^(qg  +  dt  ,  -  1  . 


The  characCeristic  function  of  x^  is  given  by: 


e^.(z)  -  exp(iz[xj.  -  ^  “  1  “ 


(1.50) 


2.  Control  and  Special  Cases 


j  exp(a  Fj^+a[F2+iz(|i^  ]  )dF(a) 

» 
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j  exp(a  Fj^+aF^jdFCa) 


Two  special  cases  of  Eqs.  (1.1)  and  (1.2)  result  in  significant  simplifi¬ 
cation  of  the  filter  equations.  The  first  case  occurs  when  gQ(t,y)  =  0, 

0  <  t  <  T.  From  (1.7)  it  follows  then  that  x^  is  of  the  form 

Xf  =  Aj.(y)xQ  +  B^(y)  . 

Using  the  above  equation  in  (1.2)  we  have  the  following  estimation  problem: 

Let  Xg  be  a  random  variable  with  distribution  F(a)  “  P(xq  <  ajyg).  Assume 
that  the  observation  process  y^.,  0  <  t  <  T,  admits  a  differential 

dFt  •  '*■  ‘^^t  ’ 

where  the  notation  stays  the  same  as  in  (1.2)  and  hg,  h^  satisfy  (1.3)  and 
(1.4). 

From  Lemma  2  it  follows  now  that  the  conditional  characteristic  function  of  Xg 
given  is  of  the  form 


(2.1) 


e^(z) 


2 

/  exp[a  Fj  +  aF^  +  Iza)dF(a) 
_  - 

/  exp(a  Fj  +  aF2)dF(a) 


The  above  results  from  the  fact  that  dxg  -  0  replaces  Eq .  (1.1)  implying 
Pj(0)  ■  0  and  x^(&,0)  *  a,  as  defined  by  Bqs.  (1.21)  and  (1.22).  Now 


<le^(z) 

dz  z-0  “  ’ 
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In  order  to  discuss  a  control  problem  uslnj;  the  results  obtained  here,  assume 
that  all  the  coefficients  In  (1.1)  and  (1.2),  except  fo(t,y)  which  Is  denoted 
here  by  u^(y),  are  functions  of  time  only.  If  u^(y)  satisfies  the  assumption 

(1.5)  we  say  that  u  »  {n^(y),  0  <  t  <  T}  Is  an  admissible  control  and  write 

u  e  n. 

Let  x^ ,  denote  the  solutions  to  (1.1)  nad  (1.42)  respectively,  for 
some  u  e  D,  and  let  x°,  x^  correspond  tou^  =0,  0  <t  <T. 

Define  Cj.  =  x“  -  x^,  Oq  =  Xg  -  x^  =  eg,  e°  *  x°  -  x°.  Substractlng  Eq . 
(1.42)  from  (1.1)  we  have 

(2.5)  de“  -  f^e^dt  +  g^dw^  +  q^dv^  -  (q^  +  P^(e'^)h^  ]  (h^e^  +  dvj  , 

where  -  P{.(e*^)  shows  that  P^  depends  only  on  e^,  0  <  s  <  t  which  Is  seen 
from  Eq.  (1.48)  rewritten  as: 


(2.6)  dF^  -  ((•j.h^(e^^dt  +  dv^  +  4.^h^I^(  1)  )dt  . 

From  (2.5)  It  follows  that  with  probability  one  the  values  of  e"  and  e° 
coincide  for  all  u  e  U.  Now,  since 

(2.7)  dv"  -  dy"  -  (hg+  h^x")dt  -  h^e^dt  +  dv^.  -  h^e°dt  +  dv^  -  dv°  , 

and  v°  coincide  with  probability  one.  Prom  (2.7)  It  follows  that  Bq . 
(1.42)  can  be  rewritten  as 


(2.8)  dx“  -  Ujjdt  +  (qQ+  ''iPtleJ)  )dv°  ,  x“  -  x^  -  /  adF(a) 


Now  let  u^  “  Fj(x  ),  where  F^  Is  a  nonantlcipative  functional  of  x'^, 
0  <  s  <  t,  and  satisfies 
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The  above  shows  that  la  measurable.  This  fact  combined  with  (2.9) 
states  that  e  0,  and  that  we  can  expect  the  separation  of  the  stochastic 
control  of  type  and  the  filtering  problem.  As  an  illustration  of  the 
statement,  consider  the  following  control  problem. 

Linear-Quadratic  Control  Problem  with  Non-Gausslan  Initial  Distributions 

The  partially  observable  controlled  process  (x^.y^),  0  <  t  <  T,  is  given 
by  the  stochastic  equations 

(2.11)  dx^  -  (fj(t)x^  +  u^jdt  +  gQ(t)dw^, 
dyt  ■  h^(t)x^dt  +  dv^  ,  y^  -  0  . 

The  independent  Wiener  processes  w^  and  v^  entering  into  (2.11)  do  not  depend 
on  the  random  variable  Xq  (the  Initial  state).  Xq  is  assumed  to  have  dlstrl- 

m 

butlon  function  F(a)  ■  ^  with  /  a^dF(a)  <  «  (finite  fourth  order 


moment) 


The  ®  o-alg{yg,  0  <  s  <  t}  measurable,  stochastic  process  Is  called 
a  control  at  time  t  and  Is  assumed  to  satisfy 

^  *  * 

0 

For  u  *  {uj.,  0  <  t  <  T},  satisfying  the  above  we  write  u  e  0,  where  U  is  the 
class  of  admissible  controls.  It  is  also  assumed  that  fj,  gQ,  h^  satisfy  the 
deterministic  version  of  (1.3)  t  (1.5). 

Consider  now  the  performance  functional 

T 

(2.12)  J(u)  -  +  /  (x^H(t)  +  u^R(t))dt) 

1  i  0  ^  t 

where  hj.  >  0,  H(t)  >  0,  0  <  R**^t)  <  const.,  0  <  t  <  T.  The  admissible  con- 

A 

trol  u  e  U  is  called  optimal  if 

J(u)  ■  inf  J(u)  . 
u  e  U 

Lemma 

The  optimal  control  for  the  process  (2.11)  and  the  performance  index 

(2.12)  exists  and  is  defined  by 

(2.13)  -  -R“^(t)Q(t)Xj.  ,  0  <  t  <  T 
where  Q(t)  >0  satisfies  the  Rlccatl  equation 

(2.14)  2f^(t)Q(t)  +  H(t)  -  Q^(t)R"^(t)  ,  Q(T)  -  h^  , 


and  X  is  defined  by 


(2.15) 


[f^(t)  -  R  ‘(t)Q(t)  jx^dt  +  Pj.(v)h^(t)dVj. 
00 

/  adF(a)  , 


dy^  -  h^(t)x^dt 


P(t)  +  <t,(t)^[l^(2)  -  1^(0) 


*  2f  ^(t)P(t)  +  ggCt)  -  hJ(t)P^(t)  ,  P(0)  =  0 


<>(t)  »  exp(  /  (f,(s)  -  h,(s)P(s))ds 
0  ^ 


F,<t)--|  /  h^(s)<|.^(s)ds 

0  ^ 


FjCt.v)  ■  /  4i(t)hj(t)  (dv^  +  4i(t)hj(t)I^(l)dt) 


/  a”exp(a^Fj(t)  +  aF^Ct, v) )dF(a) 
__ 

/  exp(a  F^(t)  +  aF2(t,v))dF(a) 


I^(n) 


,  n  »  1,2 


Remark.  The  sCrucCure  of  Che  optimal  control  law  is  Identical  with  the  opti¬ 
mal  controller  for  LQG  problem. 

Proof  of  the  Lemma 

First  note  that  Che  assimptlons  made  In  the  control  problem  statement 
assure  well-defined  filter  for  the  system  (2.11)  and  u  e  U.  Next  rewrite  the 
performance  index  as  follows: 
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T 

(2.16)  J(u)  =  K[E[x^h^|T^)  +  /  E(x^H(t)  +  u^R(t)|Yjdt] 

»  B((xli]^h  +  /  ((x“)^H(t)  +  u^R(t))dt) 

i  1  0 

T 

+  E(h  P“  +  /  pVt)dt] 

0 

Prom  (2.6)  we  conclude  that  P“  does  not  depend  on  the  control  u  and  coincides 
with  the  function  P°  obtained  from  the  filter  equations  for  u^  =  0,  0  <  t  <  T. 
The  process  x^  entering  (2.15)  satisfies  equation  (see  2.8) 

(2.17)  dx“  -  [fj(t)x“  +  u^jdt  +  hj(t)P°dv°  , 

t 

where  *  /  (dy  -  h  (s)x  as),  according  to  (2.7),  and  v°  is  a  Wiener 

C  C  Q  S  X  S  “ 

process  (see  (1*11))*  Introduce  now  the  function 

T 

(2.18)  V(t,5)  -  C^Q(t)  +  /  Q(T)hj(T){P°J^dT 


where  0  <  t  <  T,  -•  <  5  <  and  Q(t)  satisfies  (2.14).  It  is  easy  to  verify 
that  V(t,5)  satisfies  the  following  Bellman  equation; 


(2.19) 


C^H(t)  +  Cf^(t)  — j  (P°)\^(t) 


3^V(t.C)  .  3V(t.£) 
3C^ 


+  min(Ti^R(t)  +  n  “  0  , 

n 


and  that  V(T,C)  *  C^h.^* 

A 

Note  that  n  which  minimizes  the  above  for  positive  definite  R(t),  is  given  by 
(2.20)  n  -  -R~^(t)Q(t)C  . 
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Calculate  now,  with  the  use  of  the  Ito  formula, 

T 


v(T.x-?)  -  v(0.;„)  .  /  ((M|^  4 


0 


35 


5=x. 


^  3V(t.C) 


35 


5=x 


dx^ )  . 
u  t 


Taking  Into  account  (2.8)  and  (2.19)  we  obtain 

T 


V(T,x“]  -  V(0,Xq]  >-  /  ((x“]^H(t)  +  u^R(t)]dt 


0 


+  2  /  x“Q(t)P°h  (t)dv°  . 
0 


After  taking  the  expectation  of  both  sides  of  the  above  Inequality, 

T 


(2.21)  V(0,x^}  <  E((:^^)2h^  +  /  {  {x“)^H(t)  +  u^R(t)  )dt ) 

o 


The  equality  In  (2.21)  holds,  according  to  (2.20)  only  If 


(2.22)  u^  -  -R"^(t)Q(t)  xj  . 


Comparing  (2.22)  with  (2.16), 


J(u)  <  J(u)  for  all  u  e  U. 


The  admissibility  of  u  defined  by  (2.22)  follows  from  (2.10)  and  the  fact  that 


E(sup  (x^)^]  <  •  . 

0  <  t  <  T 

The  above  can  be  proven  In  the  same  way  as  in  the  derivation  of  a  conditional¬ 
ly  Gaussian  filter  [Lemma  12.1;  4,  pp.  18-19).  This  ends  the  proof  of  the 


It  seems  to  be  possible,  (following  e.g.  [2])  to  show  that  the  separation 
principle  holds  also  for  nonquadratic  performance  functionals. 


i 
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